As computer and network devices and systems continue to advance and become more complex, effective and efficient data transfer between the various components in the computer and/or network processing systems have become more and more critical in system design and implementation. In particular, considerable effort and research has been focused on various mechanisms to reduce or hide memory latency in computer and network systems. In various processing applications such as network processing applications, in addition to the bandwidth of memory devices, the latency for memory accesses is also a significant performance limiter that affects the overall system performance. High-density memory devices such as synchronous dynamic random access memories (SDRAMs) usually have high latency in memory accesses to random locations in these memory devices. Furthermore, these memory devices have a variable latency that depends on the order of memory accesses to the various memory locations.
In a system configuration in which many system components may request memory accesses to the memory devices (e.g., SDRAMs) via a memory controller, the memory latency of these memory accesses can be very high due to the demands of the various system components which need access to the memory devices (e.g., read or write access) and the order in which the various memory access requests are processed or serviced by the memory controller. For example, in a typical system or application, a memory controller may receive memory access requests from several system components or devices (also called masters herein) to access the memory. Each master typically accesses a different area in the memory from other masters. In many applications, each of these masters accesses the memory with sequential memory addresses. In such a system or application, the latency may be very high as the memory controller typically will serve these various memory access requests from the various masters on a first-come-first-serve basis and have to incur multiple cycle penalty to switch between the different memory accesses requested by the different masters. For example, a component called master A may request access to row 1 in the memory whereas a component called master B may request access to row 50 in the memory. Subsequently, master A may request access to row 2 in the memory whereas master B may request access to row 51 in the memory. In this example, the memory controller typically will process request to access row 1 for master A and then switches to process request to access row 50 for master B, switch again to access row 2 for master A and then switch again to access row 51 for master B. As a result, the latency for accessing those various locations in memory is high and the memory controller suffer multiple cycle penalty to switch between accessing different rows in the memory.